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ABSTRACT: Exoskeletons are increasingly being recognized as ergonomic solutions for work-related 
musculoskeletal disorders in the construction industry. However, users of active back-support exoskeletons are 
susceptible to various physical and psychological risks, which could be exoskeleton type-or task-dependent. A test 
bed is needed to enable deployment and assessment of risks associated with exoskeleton-use for construction 
tasks. This study aims to develop a human-in-the-loop digital twin framework for assessing ergonomic risks 
associated with the use of active back-support exoskeletons for construction work. A literature review was 
conducted to identify risks associated with exoskeletons and the technologies for quantifying the risks. This 
informed the development of a system architecture describing the enabling technologies and their roles in 
assessing risks associated with active back-support exoskeletons. Semi-structured interviews were conducted to 
identify construction tasks that are most suitable for active back-support exoskeletons. Based on the identified 
tasks, a laboratory experiment was conducted to quantify the risks associated with the use of a commercially 
available active back-support exoskeleton for carpentry framing tasks. The efficacy of the digital twin framework 
is demonstrated with an example of the classification of exertion levels due to exoskeleton-use using a 1D- 
convolutional neural network. The study showcases the potential of digital twins for comprehensive ergonomic 
assessment, enabling stakeholders to proactively address ergonomic risks and optimize the use of exoskeletons in 
the construction industry. The framework demonstrates the significance of evidence-based decision-making in 
enhancing workforce health and safety. 
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1. INTRODUCTION 


The United States Bureau of Labor and Statistics reports that the construction workforce continues to experience 
a higher rate of work-related musculoskeletal disorders compared with workers in other industry sectors 
combined. The back is one of the most affected body parts. Back-related injuries account for about 43% of all 
work-related musculoskeletal disorders (BLS, 2023). Exoskeletons are emerging as potential solutions to 
WMSDs. Specifically, active back-support exoskeletons, a class of exoskeletons, have been shown to reduce the 
risks of overexertion which is one of the triggers of back-related injuries. For example, studies have revealed a 
reduction in muscle activity (Theurel et al., 2018), discomfort in the body parts (Gonsalves et al., 2021; Kim et 
al., 2019), rate of exertion (Alemi et al., 2020; Baltrusch et al., 2021), and range of motion (Cumplido-Trasmonte 
et al., 2023) due to exoskeleton-use. These benefits are motivating construction contractors to explore active back- 
support exoskeletons for construction work. However, studies have also mentioned that exoskeleton-use on 
construction sites could trigger unintended consequences such as loss of balance or fall risks (Alabdulkarim et al., 
2019; Kim et al., 2019; Massardi et al., 2023), physical discomfort and pain (Gonsalves et al., 2023; Gonsalves et 
al., 2021), fatigue (Theurel et al., 2018), and restricting movement when climbing ladders (de Looze et al., 2016; 
Kim et al., 2019). These highlight the likely task-specificity of exoskeletons. In addition, the unintended 
consequences could also be specific to exoskeleton types (Fox et al., 2020; Kim et al., 2019). With the increase in 
commercially available exoskeleton solutions, a testbed would be beneficial to support the testing and assessment 
of the solutions for construction tasks. This has downstream implications for enabling real-time monitoring of 
exoskeleton-use during construction, which could inform strategies for reducing unintended risks. 


Sensing technologies provide opportunities to measure risks associated with exoskeleton-use (Akanmu et al., 
2020; Ogunsejju et al., 2021). Data from sensing technologies could be modeled and analyzed to extract insights 
that can be mapped to workers’ virtual replicas and controls (e.g., rating meters). This digital representation could 
be used by stakeholders (e.g., project and safety managers, and manufacturers) to develop control strategies such 
as deciding on the contextual use of exoskeletons (i.e., most suitable application tasks and duration of use), suitable 
exoskeleton types, and modifications to exoskeleton design. This digital representation is referred to as the human- 
in-the-loop digital twin which is a two-way symbiotic relationship between physical entities (e.g., workers and 
exoskeleton) and their virtual representative in which humans initiate the control (N. Zhang et al., 2022). The 
implication is that the ergonomic consequences of workers' postures while using exoskeletons can be obtained in 
real-time which can enhance their ability to control or self-manage their exposures (Ogunseiju et al., 2021). Thus, 
this paper presents a digital twin framework for assessing the risks associated with exoskeleton-use for 
construction work. A review of the literature was conducted to identify risks associated with exoskeleton-use and 
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the sensing technologies for quantifying the risks. The risks enabled the development of a system architecture to 
support the development of a human-in-the-loop DT framework that can inform decisions to support the 
sustainable use of exoskeletons in construction. Construction workers are interviewed to identify tasks that would 
benefit from active back-support exoskeletons. The efficacy of the DT framework is demonstrated with an 
example of predicting risk of exertion due to the use of an active back-support exoskeleton for one of the tasks. 
This study motivates discourse on human-in-the-loop Digital Twins for construction applications. The study 
highlights the extent to which physiological risks associated with exoskeleton-use can be predicted. 


2. BACKGROUND 
2.1 Exoskeletons for Construction Work: Risk and Assessment Techniques 


Studies have shown that exoskeletons have intended benefits and unintended consequences. For instance, 
exoskeletons are prospective innovative ergonomic interventions aimed at reducing overexertion in various parts 
of the body (Gonsalves et al., 2021). This may in turn reduce the occurrence of WMSDs among construction 
workers (Akanmu et al., 2020). There is evidence that exoskeletons can reduce back muscle activities by 23 - 35% 
(Abdoli-e and Stevenson, 2008). Other benefits revealed in the literature include reduced discomfort to the body 
parts (Alemi et al., 2020), increased productivity, financial gains, and work retention (Kim et al., 2019), and 
increased ability to lift heavier loads and perform repetitive tasks (Mahmud et al., 2022). Despite these benefits, 
there are some unintended consequences associated with exoskeletons. Exoskeletons have been known to trigger 
risks broadly classified in Table 1 as physical and psychological risks. The physical risks include joint 
hyperextension, instability and fall risk, muscle fatigue, bruising, skin and soft tissue injury, and increased 
cardiovascular demand and metabolic cost (Howard et al., 2020; Massardi et al., 2023; Theurel et al., 2018). For 
example, skin irritation or chemical burns could occur if an exoskeleton battery leaks corrosive materials to the 
user (Howard et al., 2020). In addition, due to the added weight of the exoskeleton, the user’s center of gravity 
may be significantly impacted causing balance problems and a diminished recovery rate (Alabdulkarim et al., 
2019). Gonsalves et al. (2021) showed that exoskeletons result in discomfort in the chest and thigh regions. 
Previous research has indicated that exoskeletons can redirect loading from one part of the body to another 
(Picchiotti et al., 2019). For example, during overhead work, exoskeletons reduce the muscle activity in the 
shoulder and the back of the arm but increase muscle activity in the lower back, abdomen, and legs (Theurel et 
al., 2018). Besides, usability, self-efficacy, and safety could be negatively impacted because the exoskeletons 
could get caught around wires and may affect work postures (Baltrusch et al., 2021). Exoskeletons are sometimes 
incompatible with some personal protective equipment such as safety harnesses (Gonsalves et al., 2023). Previous 
studies (Omoniyi et al., 2020; Siedl et al., 2021) have also identified psychological risks such as decreased 
situation awareness/distraction, cognitive overload, fear of the device, and overconfidence in the device. Various 
sensing technologies, such as inertial measurement units and electromyographs, have been employed to quantify 
these risks. Similarly, objective measures from the sensing technologies have been validated with subjective 
assessment instruments such as the NASA Task Load Index and Berg Balance Scale. These are highlighted in 
Table 1. 


Table 1: Risks and assessment techniques of exoskeletons. 


Categories of risks Risks Objective Assessment Subjective Assessment Related Studies 


Joint hyperextension Inertial measurement units; Local Perceived Pressure (Theurel et al., 2018) 


Cameras scale; Borg Rating of 
Perceived Exertion scale 
Instability and Fall Pressure insoles; Force Berg Balance Scale (Alabdulkarim et al., 
risk plates 2019; Kim et al., 2019; 
Massardi et al., 2023) 
Muscle fatigue Electromyography Borg Rate of Perceived Pain (Theurel et al., 2018) 
Physical risks Scale; Borg Rate of Perceived 
Exertion Scale 
Hygiene Biocompatibility tests Usability questionnaires e.g., (Howard et al., 2020; 
issues/Bruising, Skin System Usability Scale Massardi et al., 2023) 
and soft tissue injury 
Cardiovascular Electrocardiogram; Workload assessment (Moyon et al., 2018; 
demand Photoplethysmogram questionnaires e.g., NASA Theurel et al., 2018) 
Task Load Index (TLX) 
Metabolic cost risk Indirect calorimetry Questionnaires for workload (Alemi et al., 2020; 


assessment (e.g, NASA Baltrusch et al., 2021) 
TLX), Rating perceived 
exertion (RPE) with the Borg 
Category Ratio (Borg CR-10) 
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Usability of the Eye tracker; Focus groups; Usability (Gonsalves et al., 2021; 
device (e.g., | Electromyography questionnaires; Borg CR 10 Kim et al, 2019; 
perceived discomfort, scale; Body part discomfort Ogunseiju et al., 2022) 
chest pain, catch and scale 
snag risks) 
Decreased situation Eye tracker NASA-TLX (de Looze et al., 2016; 
awareness/ Delgado et al., 2020) 
Distraction 
Exertion Electrodermal activity Rating perceived exertion (Man et al, 2022; 
Psychological sensor (RPE) with the Borg Category Theurel et al., 2018) 
risks Ratio (Borg CR-10) 
Fear; Lack of trust Electromyography; Interview; Self-developed (Omoniyi et al., 2020; 
Photoplethysmogram; questionnaires; subjective | Upasani et al., 2019) 
Electrodermal activity psychological impact test 
sensor 
Cognitive overload Electroencephalography; NASA-TLX; MF (M-VAS)  (Cumplido-Trasmonte et 
Eye tracker; Electrodermal and boredom (B-VAS) al., 2023) 
activity sensor 
Overconfidence Camera Videos; Optical Focus groups; Questionnaires (Baltrusch et al., 2021; 
effect tracking system (OTS) (e.g., Modified Spinal Siedl et al., 2021) 


Function Sort) 


2.2 Digital Twin for Ergonomic Risk Assessment 


In recent years, there have been increasing explorations of DT for diverse applications including workforce health 
and safety. Sharotry et al. (2022) developed a DT to track the biomechanical fatigue in operators caused by 
repetitive action in lifting activities. The study analysed changes in the joint angles in workers’ body joints using 
a dynamic time-warping algorithm. Another study (Greco et al., 2020) presented an ergonomic risk mapping of 
DT workstations using a wearable motion capture system and inputting in virtual simulated environments. With 
the DT, the authors were able to identify risk indexes related to working postures, exerted forces, material manual 
handling and repetitive actions, and sources of biomechanical overload. In construction, Ogunseiju et al. (2021) 
developed a DT framework for improving self-management of ergonomic risks. However, scarce studies have 
explored the assessment of exoskeleton-use using a DT environment. Furthermore, Greco et al. (2020) opined that 
existing DT frameworks have limited roles for users or stakeholders. Humans play vital roles in workplace 
systems; therefore, supporting technologies should be designed to facilitate their input (Sharotry et al., 2020). 


3. METHODOLOGY 


The approach employed in conducting this study is shown in Figure 1. First, a review of the risks associated with 
exoskeletons and the technologies for measuring the impact of the risks was conducted (Section 2.1). This 
informed the development of an architecture of a human-in-the-loop DT system for assessing risks associated with 
active BSEs. Semi-structured interviews were conducted to identify construction tasks that would benefit from 
the use of active BSEs. A laboratory experiment was conducted to quantify the risks associated with using active 
BSEs. This informed the development of an example of a DT-based model for assessing the risk levels of exertion 
during exoskeleton-use. These are described as follows: 


* Noise & Artifact Removal 
* Data Labelling 
* Data Augmentation 


Fig. 1: Overview of research methodology. 
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3.1 System Architecture 


The system architecture shown in Figure 2 is built to illustrate the proposed human-in-the-loop DT framework. 
The system architecture shows the enabling technologies and their role in supporting the assessment of ergonomic 
risks associated with exoskeleton-use. The architecture comprises six layers include physical layer, data layer, 
data transmission layer, storage layer, application layer, and access layer. These are described as follows: 


3.1.1 Physical layer 


The physical layer comprises sensing technologies and physical devices. The sensing technologies support 
capturing of physical and psychological risks, and environmental characteristics of work areas. The physical risks 
include local muscle fatigue, fall risk, joint hyperextension, and metabolic risk which can be measured using 
electromyography (EMG), pressure insole, inertia measurement unit (e.g., comprising of accelerometers, 
gyroscope, and magnetometer), and calorimeter respectively. The psychological risks include cognitive overload, 
lack of trust, and decreased vigilance which are measured using an electroencephalogram, electrocardiogram, 
photoplethysmogram, and eye tracker respectively. The workspace or site conditions can be captured using 
environmental sensors such as temperature and humidity sensors and image-based sensors such as cameras and 
laser scanners. Physical devices include reality technologies such as virtual and augmented reality devices, and 
other data acquisition technologies for collecting subjective data to evaluate the aforementioned objective 
measures. Reality technologies support the development of risk-free simulated construction site environments 
where workers can practice work with different exoskeletons. 


3.1.2 Data layer 


Data from the physical layer are captured in the data layer. The data layer contains the data generated from the 
sensors and physical devices, such as raw acceleration and angular velocity from the IMU, brain waves from EEG, 
electrical conductance of the skin from EDA sensors, eye fixations from eye trackers, muscle activity from EMG, 
and temperature and humidity from temperature and humidity sensors. Subjective data (such as perceived 
cognitive load, rate of exertion, and discomfort levels) are also stored in the data layer. This layer also contains 
videos of construction work and general characteristics of the work area that might explain or influence risk factors 
of WMSDs. 


3.1.3 Data transmission layer 


The data transmission layer transfers data from the data layer to other layers for storing, modeling and analysis, 
and DT representation. Different communication technologies could be used in this layer, such as short-range 
transmission technologies e.g., Wi-Fi, Bluetooth, Zigbee, near-field communication (NFC), and Zwave, and long- 
range transmission technologies e.g., 3G, 4G long-term evolution (LTE), and low-power wide-area networks. 


3.1.4 Storage layer 


This layer consists of cloud services that store data received from the data transmission layer and application layer. 
Heterogenous data from these layers are gathered and stored in a cloud storage system for exchange or sharing 
with other layers. The data or information can be beneficial for extracting other insights that can help improve the 
health and safety of workers. Depending on the stakeholders and their information needs, multiple repositories 
may be included. As such, different access rights may be provided. For instance, a data analyst may need access 
to label subjective data obtained from the data layer to enable assessments involving risk classifications. A 
safety/health manager may be provided access to data that can inform impact on workers’ health while impacts 
while a project manager may be provided access to data relating to impact on productivity. 


3.1.5 Application layer 


This layer includes algorithms and applications for processing and analyzing data obtained from the storage layer. 
The data are processed and represented in formats that can be used by decision-makers in the access layer (Section 
3.1.6) for decision-making. For example, to assess workers’ levels of exertion from their electrodermal activity 
(EDA) signals, this layer will use signal processing algorithms, feature extraction, and deep learning networks 
(e.g., conditional generative adversarial network, recurrent neural network, and long short-term memory), and 
visualization algorithms. Signal processing algorithms such as discrete wavelet transforms and adaptive predictor 
filtering methods will be used to reduce artifacts from the EDA signals. A symmetric multilayer perception model 
for extracting features will be used to extract informative features from the EDA signals. The extracted features 
will be fed into deep learning networks (e.g., conditional generative adversarial network, recurrent neural network, 
and long short-term memory) to classify the EDA signals into the levels of exertion. Finally, visualization 
algorithms will be used to augment the levels of exertion on a virtual replica of the worker and a rating meter. 
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3.1.6 Access layer 


In the access layer, stakeholders can visualize the impact or extent of the risks as a virtual replica of the worker 
and a rating meter. This layer includes the following: (1) beneficiaries of the DT platform and (2) how they access 
the DT platform. The beneficiaries may include safety managers, project supervisors, and product manufacturers. 
Safety managers may want to understand if the workers are reaping the intended health benefits of the technology. 
Project supervisors may want to know the impact of the technology on project performance. Both stakeholders 
could use the feedback to work with manufacturers to plan more suitable designs for their projects. The 
stakeholders can monitor the performance of the workers via interactive dashboards and web applications. The 
performance of the workers will be shown in the form of their virtual replica and a rating meter to interpret the 
risks. For example, the levels of exertion associated with an exoskeleton (e.g., no exertion, low exertion, medium 
exertion, and high risk) that is computed in the application layer, will be shown as different colors in a virtual 
replica and rating meter (e.g., green, yellow, and red human). In this way, the project stakeholders can understand 
the type and extent of the risk, which could inform decision making such as which type of exoskeleton to use for 
what task, how long the exoskeleton should be used for the task, and changes that should be made to the device 
to better adapt it to construction work. 


Access 
Layer 
Stakeholders Exoskeleton users 
Application = D| SR 
Layer V ss Signal processing 
isualizati imizati i algorithm 
Storage 
Layer 
Historical database Local database Database server 
Data 
`ransmission 
Layer 
Data 
Layer 
Sensing technologies Physical devices 
Layer 
LS E Virtual reality Exoskeletons 


Fig. 2: System architecture. 


3.2 Semi-Structured Interview 


Semi-structured interviews were conducted with industry practitioners (n=8) to understand the construction tasks 
that would benefit from the use of active back-support exoskeletons. A purposive sample was used to identify and 
select potential participants who could provide valuable insights to the study. The research team selected 
participants with experience in construction safety, and technology implementation in the construction industry. 
The interviews were conducted over Zoom and recorded. The transcripts of the interview were coded and 
emerging themes were identified. An inter-coder reliability test was conducted on the coded data using the Cohen- 
Kappa coefficient. Cohen-kappa coefficient of 0.90, indicating a strong level of agreement, resulted from the 
assessment. 
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3.3 Experimental Procedure, Participants, and Data Collection 


Sixteen students were recruited to participate in a carpentry framing activity; one of the activities identified from 
Section 3.2 as beneficiaries of active back-support exoskeletons. The participants reported no prior issues related 
to musculoskeletal disorders that could affect their performance in the study. The experiment was approved by the 
Virginia Tech Institutional Regulation Board (IRB: 19-796). The task involved the following: (1) measuring 
timber planks (i.e., four 1”x4”x47” planks and two 1”x4”x70” planks) needed to construct a 47”x70” frame; (2) 
assembling the measured timber materials as shown in Figure 3; (3) nailing the assembled timber frame using a 
nail gun; (4) lifting and moving the erected frame, which weighs approximately about 40lbs, to an upper floor via 
staircase for installation on the upper floor; and (5) installing the frame by aligning the frame with an existing 
wall. The participants completed these tasks while wearing CrayX, an active back-support exoskeleton from 
German Bionics. Their electrodermal activity was measured using Emotibit, an open-source biosensor. The data 
was collected at 50Hz i.e., 50 data points per second. After performing the framing task, each participant was 
presented with Borg’s rating of exertion scale (Borg CR-20) and asked to provide subjective ratings of their 
perceived exertion for the entire task. The Borg scale ranges from 6 (not exhausting) to 20 (extremely exhausting) 
(Albert et al., 2021; Borg, 1982). The participants’ ratings of perceived exertion were measured using the Borg’s 
exertion rating scale (Borg CR-20) which ranges from 6 to 20, where 6 represents no exertion and 20 represents 
maximum exertion. Figure 3 shows participants performing framing tasks with the exoskeleton and the biosensor. 
The task was video recorded. 


EEG cap 


Fig. 3: Participant performing framing task while wearing an exoskeleton and EEG cap. 


3.4 Data Preprocessing 
3.4.1 Noise and artifact removal 


The collected EDA signals were filtered to remove the noise and artifacts. A low pass filter with a cut-off frequency 
of 4 Hz was employed to remove the noise. A Gaussian filter was used to smoothen and attenuate the artifacts. 
MATLAB was used for this purpose. 


3.4.2 Data labeling 


Using the time-stamped video, EDA data corresponding to each participant’s tasks were sorted and structured. 
The ratings of 7— 11, 12 — 14, and 15 — 20 were represented as low exertion, medium exertion, and high exertion, 
respectively (Chowdhury et al., 2019). The sorted EDA data of each participant was labeled based on their 
intensity class as shown in Table 2. 


Table 2: Classes, labels, and data points. 


Classes Labels Percentages of participants (%) Number of data points 
Low Exertion LE 70 50759 
Medium Exertion ME 12 6823 
High Exertion HE 18 14072 


3.4.3 Data augmentation 


The EDA data of the minority classes were augmented due to the imbalanced nature of the datasets. Studies have 
shown that balanced datasets result in the Synthetic Minority Oversampling Technique (SMOTE) being employed 
to balance the EEG data of the minority classes with the majority classes (Sowjanya & Mrudula, 2023). For 
example, from Table 2, the ratio of the datasets in the LE, ME, and HE classes (i.e., Low Exertion, Medium 
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Exertion and High Exertion respectively) is 7:1:4. As shown in Table 3, more datasets were generated with 
SMOTE to balance the datasets of the minority classes (i.e., Medium Exertion and High Exertion) with the class 
with the most datasets (i.e., Low Exertion). 


Table 3: Classes, labels, and data points (raw and balanced). 


Classes Labels Number of raw data points Number of balanced data points 
Low Exertion LE 50759 50759 
Medium Exertion ME 6823 50759 
High Exertion HE 14072 50759 


3.5 Risk Classification 


This study employs a 1-D convolutional neural network to classify the EDA data into the above-mentioned classes 
(1.e., LE, ME, and HE). 1-D CNN is suitable for 1D signals whose applications have high signal variations (Y. 
Zhang et al., 2022). The network comprises an input layer, a convolution 1-D layer, a batch normalization layer, 
a Rectified Linear Unit (ReLUV) layer, a dropout layer, a maxpooling layer, a fully connected layer, a softmax layer, 
and a classification layer. The convolution layer applies filters to the input data obtained from the input layer and 
extracts distinctive features using 10 filters of width 10. The batch-normalization layer improves the stability and 
speed of training the network by normalizing the input to each layer. The ReLU layer applies a non-linear 
activation function to the output of the batch-normalization layer. The dropout layer helps to prevent overfitting 
of the model. The maxpooling layer down-samples the output of the dropout layer. In the fully connected layer, a 
linear transformation is applied to the input vector through a weight matrix so that every input influences every 
output of the output vector. The softmax layer takes in the output from the previous layer and presents a vector 
that illustrates the probability of the class that the input belongs to. The classification layer presents the results of 
the softmax layers as classes of the assessed risks. The network was trained using the Adam optimizer (Karim et 
al., 2019). Due to the size of the dataset, 300 epochs were used. The learning rate was set to 0.01. 


The balanced data was split as follows: 70% of the data was set aside for training, 15% of the data was also set 
aside for validation and 15% was intended for testing the trained model. MATLAB R2023a, installed on a machine 
with NVIDIA GeForce RTX 2080 GPU and 16GB memory, was employed for the classification. Commonly used 
metrics for assessing the performance of machine learning models were employed in this study. These include 
accuracy, precision, recall, and F1-score (Bangaru et al., 2021). 


4. RESULTS AND DISCUSSION 
4.1 Construction Tasks for Active Back-Support Exoskeleton-Use 


The results of the semi-structured interview were represented in the form of a word cloud. Word clouds are 
graphical representations of the frequency of concepts or keywords that are significant in discourses (Adu, 2019). 
The word cloud in Figure 4 provides a quantitative and visualized method to illustrate the key construction tasks 
suggested by the participants to benefit most from active back-support exoskeletons. The most mentioned tasks 
include plumbing, carpentry, steel, drywall and rebar installation, and labor work. The least mentioned tasks 
include ceiling, electrical, scaffolding, and flooring work, mason, and ceiling work. The suggested tasks support 
reporting of industry databases (BLS, 2023) and research studies (Antwi-A fari et al., 2023; Gonsalves et al., 2023) 
that identified back-related injury as a concern in the construction industry. Some of the tasks suggested by the 
practitioners (e.g., carpentry work, rebar installation, concrete work, and masonry) were also identified by (Kim 
et al., 2019) as being suitable for exoskeletons. Similarly, Gonsalves et al. (2023) identified framing and plumbing 
as being more suitable for back-support exoskeletons. 
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Drywall lab 
Masonry 
Plumbing °" 


Roofing 


Electrical Painting Ceiling 
Concrete 


Carpentry Steel 


Flooring Rebar 
Scaffolding 


Fig. 4: Word cloud of construction tasks that would benefit from active back-support exoskeletons. 


4.2 Example of Prediction of Level of Exertion from Exoskeleton-Use 


This study presents an example of predicting the levels of exertion due to exoskeleton-use. This section presents 
the performance of the 1-D convolutional neural network in classifying the levels of exertion during exoskeleton- 
use for a framing task. 


4.2.1 Model performance evaluation 


The accuracy of the 1D-CNN in classifying the levels of exertion due to exoskeleton-use for the framing task is 
82%. The confusion matrix for the model is illustrated in Figure 5. The matrix shows that the model performed 
better in detecting the ME and HE classes than the LE class. For example, the model detected classes ME and HE 
with 100% accuracy and LE class with 67%. Furthermore, 37% of the LE class is mostly confused with the HE 
class. 


LE ME HE 


33% 
LE 


HE 


True Class 


Predicted Class 
Fig. 5: Confusion matrix showing classification accuracies of levels of exertion due to exoskeleton-use. 


From the precision (see Table 4), it can be observed that out of the times ME and HE classes were predicted, the 
model was correct 100% of the time. However, out of the times HE class was predicted, the model was correct 
67% of the time. For the recall, out of all the times the HE class was predicted, only 75% of the class was correctly 
predicted. 


Fl-score explains a model’s ability to both capture classes (recall) and accurately capture the classes (precision). 
The F1-score of the ME class is 100%, meaning that the model has a balanced ability to accurately capture all the 
ME classes from the data. However, in the case of the Fl-scores of the LE and HE classes which are 80% and 
75% respectively, the model will have a mixed reaction. For the LE class, while the model may be correct 67% of 
the time, all the LE class predicted will be correct. The findings of this study have shown the effectiveness of 1D- 
CNN in classifying EEG signals (Alzahab et al., 2021). The lowest Fl-score obtained in this study is still high 
and can be compared to other construction-related studies (Xiong et al., 2022). 


Table 4: Performance metrics for ID-CNN for the classes. 


Classes Precision Recall Fl-score 
LE 67% 100% 80% 
ME 100% 100% 100% 
HE 100% 60% 75% 
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4.2.2 Level of Exertion and Digital twin 


The digital twin of the exoskeleton-users and the rating meter (shown in Figure 6) shows the level of exertion 
resulting from the results of the model. The digital twin shows an exoskeleton-user experiencing medium exertion. 
The meter comprises a pointer and three different colors, red, yellow, and green indicating high, medium, and low 
exertion respectively. The pointer reflects the level of exertion which is currently shown as medium exertion. 
Related studies have shown the possibilities of creating similar interfaces for data acquisition as a human-in-the- 
loop digital twin (Locklin et al., 2021). This exoskeleton risk assessment dashboard can also be extended to show 
the muscle activity, cognitive load, and fall rating of the exoskeleton user. This vital data collected via sensors 
can be useful for supervisors and managers to monitor workers while using the exoskeletons. 


Exoskeleton Risk Assessment Dashboard 


Muscle Activity Cognitive Load Exertion Fall 


J Low Exertion I Medium Exertion [JJ High Exertion 
Low Exertion 


Medium Exertion 


High Exertion 


Fig. 6: Dashboard showing digital twin representation of the level of exertion. 


5. CONCLUSIONS, LIMITATIONS AND FUTURE WORK 


This study aims to investigate a digital twin framework for assessing the risks associated with exoskeleton-use for 
construction work. A review of the literature was conducted to identify risks associated with exoskeleton-use, and 
objective and subjective methods for assessing the risks. A system architecture was developed to illustrate the 
enabling technologies and their roles in supporting the proposed framework. Results of interviews with 
construction workers identified carpentry framing task as one of the construction tasks that can benefit from active 
back-support exoskeletons. Electrodermal signals were collected during the experimental simulation of the 
framing task with an active back-support exoskeleton. 1D-CNN trained to classify electrodermal data 
demonstrates the potential of the DT framework to predict the exertion levels of exoskeleton users during framing 
tasks. This study contributes to the scarce literature regarding the use of digital twins for assessing the suitability 
of exoskeletons for construction work. The study demonstrates the role of physiological sensing and machine 
learning techniques in facilitating the implementation of the digital twin framework. Furthermore, this study sets 
precedence for research involving the use of digital twins for performance monitoring of exoskeletons during 
construction work. Such efforts could promote the sustainability of exoskeleton solutions in the construction 
workplace. This study had some limitations which are currently being addressed in an ongoing study. Firstly, 
EDA data was generated from students engaged in a laboratory-based simulation of framing tasks. The use of 
experienced construction workers could produce data that can help develop prediction models that are 
generalizable for the construction population. Secondly, only the exertion levels using the EDA data was modeled 
to demonstrate the digital twin framework. Further work will involve other sensing technologies and the prediction 
of other physical and psychological risks. Future studies will also involve a user assessment of the digital twin 
framework with intended users. 
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